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ABSTRACT 

In this speech, the author explains that 
metamethodology is a system designed to develop and test a 
methodology for a specific, definable purpose; and, as such, to 
provide for the development of, and the research into, methodologies* 
The paper begins with a brief clarification of the difference between 
methods and methodology and a discussion of the development and 
refinement of metamethodology. The main body of the document is 
concerned in detail with the seven basic steps of metamethodology: 
(1) State the purpose of the proposed methodology; (2) test the 
purpose by criteria such as its desirability, practicability, 
uniqueness, and operationability; (3) analyze xhe implications of the 
purpose; (4) operationalize the purpose; (5) design procedures; (6) 
test the procedures; and (7) revise the purpose and/or procedures, if 
necessary. The author contends that the steps should accomplish three 
things necessary to produce the best possible methodology for a 
definable purpose: (1) The determination of the purpose, (2) the 
development of the steps that make up the methodology, and (3) the 
testing of the methodology to see that it accomplishes the purpose. 
The complete methodology of metamethodology is shown in an appendix. 
(Author/DN) 
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Methods and methodologies have been developed over the years to do 
many different things. Scientists ~ behavioral and physical ~ Engineers, 
Businessmen, laborers and even Teachers used methods to accomplish their 
purposes. Through the use of methods jobs are made easier to do, and 
better more consistent work is done. 

For example, the physical scientists have the "Scientific Method" for 
doing research and establishing the results as knowledge. Any research 
that violates these methods is not accepted as valid by the particular 
scientific community. Another example of the use of methods comes from the 
field of Education. In order to earn a certificate to teach in either 
primary or secondary schools, a student usually has to take prescribed 
methods courses such as methods of teaching science, social studies, math 
and English. These courses usually attempt to show the students how to 
impart the subject matter to their students;. Methods, good methods or 
bid methods, are constantly used by teachers. 

There are many more examples of methods and methodologies being used 
or needed. In general there doesn't seem to be any field, job or area 
of endeavor that does not lend itself to the use of methods. For example, 
in the past ten years a new field has been created. This field is Futuristics. 
When the different aspects of this field were being^^ explored, one. of the most 
prominent divisions, where there was. and is a .great need, is the area of 
methods. This division supplies those things which are necessary in order 
to do futuristics. 

Education, right after the Russians put Sputnik into orbit, heard a 
great call for more and better scientists in all areas of the physical 
sciences. In response to this call new curricula in physics, biology, math 



and other fields were developed and disseminated. These curricula, not 
only included the subject matter, but also included methods to get across 
the subject matter. For example, PSSC physics emphasized the use of 
the lab to help the students learn the subject matter.. 

There is a difference between methods and methodologies. Methods are 
rules or procedures that guide someone in accomplishing a purpose. Methods 
consist of "rules of thumb" or "guidelines". Methodology, on the other 
hand, is a series of operational steps that accomplish a specific, definable 
purpose. The difference is that a methodology provides a specific, well- 
defined route that accomplishes the purpose while the method only supplies 
a possible route that is not well defined. A method only supplies direction 
to the user and leaves a lot for the user to supply; a methodolgy attempts 
to supply as much as possible to the user as far as operational procedures 
and sequence are concerned. 

In the previous examples, one is dealing with methods rather than 
methodologies. The "Scientific Method" does not meet the definition of 
methodolgy because it does not present a series of operational steps, but 
a general set of steps that only gives the user the main steps in doing 
research. Teaching methods are only generalized approaches to teaching. 
At no time does a teaching method prescribe a specific behavior that the 
teacher should use in a specific situation. A methodology attempts to fill 
in all the missing pieces and thereby be able to prescribe what behavior 
is needed when. 

Furthermore, a methodology can be looked at as an abstract but 
operational solution to a class of problems. It is abstract because it 
does not supply a specific solution to a specific problem but it supplies the 
means by which that specific problem is derived. It is operational because the 
steps by which the specific solution is determined are as perscriptive as 



possible. A methodology deals with a class of problems. Any specific 
problem has particular characteristics that makes it similar to other 
problems. The steps of a methodology are designed on the general problem. 
In application, by accounting for the particular circumstances, a speciTic 
situation is designed for a specific problem. It is in this way that a 
methodology is an abstract but operational solution to a class of problems. 

The need for methodologies has never been strongly perceived. This 
could be because methods are so much a part of what we do that we take them 
for granted. But the need is there and it is strong. With the way things 
keep changing either new or improved methods are needed. Occasionally this 
need is strongly perceived as happened in Education after Sputnik. 

But in this proliferation of methodologies there has never been a 
methodblogy that provides for the development of, and research into, 
methodologies. In the past, any person who wanted to develop methodologies 
simply depended on his, intuitive understanding of methodologies and his 
creative abilities. Given the low perception of need, and the fact that 
any method is better than none this lack of a conscious methodology for 
the creation of methodologies never appeared to be a hinderance. As a matter 
of fact in this absence a type of engineering came about whose practitioners 
were actually developers of methodologies. This field is industrial engineering. 
An industrial engineer develops methods to produce a better product in a more 
efficient way, thereby optimizing as much as possible the use of available 
resources. 

Certain occurances have pointed to the need for a methodology to 
develop and research methodologies. These occurances include the need for 
an effective Evaluation Methodology and a Client Demand Methodology. The 
need for an Evaluation Methodology based on the purpose to provide data 
for decision-making has been documented by Larry Benedict (U.Mass. 1971). 
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The need for a Client-Demand Methodology based on the purpose to determine 
client demand for public services has been documented by Richard Coffing 
(U.Mass, 1971), 

In attempting to fill the need for an Evaluation Methodology, the 
Fortune-Hutch inson Evaluation Methodology has been conceived and is being 
developed by Dr, Jimmie Fortune, Thomas E, Hutchinson et.al. In 
attempting to communicate how to develop and research this Evaluation 
Methodology the lack ot an effective methodology to develop and research 
methodologies was perceived. This became even more evident when one attempted 
to learn or to ;each how to develop and research methodologies. 

It was to fill this gap that Dr. Hutchinson conceived of the concept 
of Metamethodology. This methodology has the purpose to develop and test 
a methodology for a specific, definable purpose. The first step taken was 
the conceptualization of the seven basic steps of Metamethodology. These 
were determined by Dr. Hutchinson and presented first by Richard Coffing 
in his dissertation proposal (U.Mass. 1971) which was concerned with the 
development of a Client Demand Methodology. These seven steps are: 

1) State the Purpose 

2) Test the purpose by criteria such as 

a. Is it desirable? 

b. Is it pperationalizable? 

c. Is it practicable? 

d. Are existing methodologies insufficient? 

3) If the answers are affirmative, then analyze the implications 
of the purpose. 

4) Operationalize the purpose 

5) Design procedures 

6) Test the procedures 

7) Revise the purpose and/or procedures, if necessary. 
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The next step of development came when' Dr. Hutchinson and James 
Thomann decided to develop the methodology further. The reason for this 
undertaking was the desire of James Thomann to be able to develop and test 
methodologies and also be able to teach others to do the same. Since the 
above seven steps were all that existed of Metamethodology, it was determined 
that the development of Metamethodology was necessary in order to train 
other methodologists . 

A two-part process was chosen to develop Metamethodology, First, the 
two developers decided to use the existing steps to develop a methodology 
on a given purpose, \flierever there was no specific procedure spelled out 
in Metamethodology, the developers would document as best they could the 
things they did to accomplish that particular step. After some study, both 
of areas of interest and need for methodology in these areas, the area of 
Futuristics was chosen and the purpose determined for the methodology was to 
provide information and data to decision makers on the consequences of the 
alternatives they face. 

The second part of the development: process was to use the existing 
steps of Metamethodology to fill in gaps in the Metamethodology itself. 
This was a process of using what existed of Metamethodology to develop 
itself. In this process the circular nature of Metamethodology is easily 
seen. This entire combined process has been compared to the process of 
evolution and because of its success, the developers have tried to make it 
an integral part of Metamethodology. This part is a combination of Field 
Test and Conceptual Development. 

Five subsequent drafts of Metamethodology have been written since the 
first seven steps. In addition, there have been two drafts of the Future's 
Methodology produced. The sixth draft of Metamethodology is described in the 
following pages and the complete methodology (Draft I) is attached as an 
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appendix. Also attached as an appendix is the latest draft of the Future's 
Methodology. 

Metamethodology has changed somewhat in its basic steps ♦ There are still 
seven steps but through combination a number of steps have been put together 
and a couple of others added. Furthermore, all the steps have been expanded. 
It would be a mistake to say that Metamethodology is complete. There are 
still gaps to be filled, but the basic makeup of Metamethodology appears to 
be complete and only the further operationalization of the steps seem to be 
needed. 

Previous3.y it was iinentioned that a methodology is an abstract but 
operational solution to a class of problems. Given that this statement is 
fact, then Metamethodology is an abstract but operational solution to the 
class of problems: all definable problems. The class of problems is all 
definable problems since Metamethodology provides for the development and 
testing of methodologies for any class of definable problems and therefore 
is- a solution for all definable problems* The one constraint on Metamethod- 
ology is that the class of problems must produce a definable purpose, which 
when accomplished solves the problem* 

There are three things that are necessary to produce the best possible 
methodology for a definable purpose: 1) the determination of the purpose; 
2) the development of the steps that make up the methodology; 3) the testing 
of the methodology to see that it indeed accomplishes the purpose. In the 
seven steps Metamethodology (Draft VI) accomplishes the three things listed 
above. What follows then is an explication of the seven steps of Metamethodology* 
Each step will be described conceptually, but no attempt will be made to 
totally describe each step since the complete methodology is an appendix 
to this paper* 

The first step is to put the methodologist into contact with the 
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problem. This step identifies in one of two ways the area in which a 
methodology is needed. The simple way i s to use the interests of the 
methodologist and the complex method is to do a Client Demand Study using 
Coffing's Client Demand Methodology. 

Step II is to determine the purpose around which a methodology is to 
be developed. This is accomplished by doing as thorough an investigation 
of the problem area as is possible. In doing this investigation, the nature 
of the problem area to be determined. By determining the nature of the 
problem area one has begun to identify what it means to work in the area. 
From this process, one can then determine, a purpose for which to build a 
methodology in order to solve the problem. At this writing this step is one 
of the least developed steps of Metamethodology ♦ There is no process of 
investigation that the developers feel is superior to any other* For that 
matter, no specified process yet exists for this activity. 

In step III the purpose is tested against four criteria. The first ■ 
criteria is desirability. By this criteria, one attempts to d^ dne if 
the methodology developed around the purpose will accomplish soinfc..ning 
people want and will use. For if the purpose is undesireable then producing 
a methodology that will accomplish this purpose might be a waste of tinie. 

Operationability is the second criteria. By this criteria, it is determined 
if the purpose can be made operational and thereby be totally understood. 
It is not necessary to operationalize the purpose at this time, but- only 
determine if it can be made such since an operational .purpose is necessary 
for later stages of the methodology, and since a prupose that is not opera- 
tional may be unsolvable. 

Next, one determines if the purpose is practicalbe. Practicability, 
first, calls for a determination as to whether a methodology can be developed, 
Y^jJ^^ given the resources available for the development. It might be unwise to 

L— begin work on a methodology when there are not sufficient resources to 
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complete the developmental tasks. Secondly, practicability calls for the 
determination as to whether the methodology implied by the purpose can be 
applied practically, once it is developed. If the methodology cannot be 
applied practically then there is a good chance it can not be used or will 
not be used^ 

The final criteria against which the purpose is cested is the insuf- 
ficiency of existing methodologies for the accomplishment of that purpose. 
This criteria is used to make sure that time and resources are not wasted 
developing and testing a methodology for the chosen purpose that does the 
same thing in a way it accomplishes the purpose or in that it does not do a 
better job of accomplishing the purpose than existing methodologies designed 
for the same purpose. This criteria can also help save time and resources 
by identifying gaps in the existing methodology. If any of the above 
criteria do not test positively then the purpose is reworked or all work on 
the methodology halted, depending on the extent of the risk, and resources 
available to the methodologist. 

The fourt step of metamethodology is designed to produce the skeleton 
outline of the methodology. After the completion of this step one can have a 
pretty good idea of waht the final , me thodolgy will look like. First, the 
methodologist analyzes the implications of the purpoi^e and then organizes 
these implications into a rational order of steps. This is done because it 
produces the first approximation of the gross methodological elements, for as 
Dr. Hutchinson said, "Every problem implies its own solution^** (1971) ^_The 
methodologist would then add in any necessary steps that are on the same 
level of operationalization, but were not part of the implications. This is 
done because there is no guarantee that the implications will produce the 
entire skeleton. For example, transitional steps might be needed in order to*" 
ERiC ""^^^ methodology workable. Finally, the very first and very last steps 

„Ma determined and added the methodoJ.ogy_if they are not^already^there.. 
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Next, the methodologist operationalizes the purpose if it was not 
" done in step III above. This is necessary in order to carry out the last 
two steps of Metamethodology. Since the last two steps provide for the full 
development and testing of the methodology, objective criteria are needed 
against which to judge and test the methodology. By operationalizing the 
purpose the methodologist procludes the necessary criteria. This is why it 
is so important to test the purpose for operationalizability, since otherwise it 
would be difficult to produce the necessary criteria at this step. 

Step VI provides for the further design of the methodology. Through 
this step at least one, if not most, of the gaps of the methodology are 
filled. The step is divided into two basic sections with a recycling component. 
The first part i* to identify a gap (gaps) and design the steps to ^'ill it. 
These sufesteps are designed by determining a subpurpose to fill the gap and then 
by analyzing the implications of the subpurpose the substepa are developed. 
The second part of the step provides for a logical testing of the newly developed 
substeps in terms of their internal logic of the developed substeps and in 
terms of the whole methodology by using the criteria produced in step V. It 
is important that both logical tests are passed, since it can not be just 
assumed that the newly developed steps will be logically consistent. The 
recycling component provides for the steps under development to got through 
redesign until they appear to satisfy the criteria. And it also provides 
for the methodology to be recycled until either all the gaps are provided 
for or until the methodologist feels he cannot sufficiently improve the 
methodology to warrant using any mL "s resources on this step. 

Finally, Metamethodology provides for field testing and conclusion- 
oriented research cf the methodology. A field test is a controlled use 
of the methodology that provides data for further design or redesign of 
O parts of the methodology. Conclusion-'oriented research is the testing of 

ERIC 

h r iii riiiwnfTiiiii i in hypotheses about the methodology. Again, these are done in terms of the 
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criteria produced in Step V, This step also has a recycliug component, ihe 
recycling puts the methodology back into sC3p VI to solve the problems 
identified by the testing or research. 

At this point in time there is no rigidity in the order of steps • 
For example, Step V can be done ^hen it is needed since some methodologists 
might find it more appropriate to do this step earlier or later than specified. 
Even though rigidity is not there, it is recommended that thci methodologio 
follow the methodology unless his experience determines a better way. One 
reason that this lack of rigidity exists is because Metamethodology is 
still under development. 

Furthermore, it should be noted that Stepr> VI and VII can be going on 
simultaneously. This can be done because step VII can help the methodologist 
identify the gaps and step VI provides steps that can be tested by c^tep VII 
to assist in the development of these steps. Research, either field testing 
or conclusion-oriented, can be done on anv pa::t of the methodology as well 
rs on the whole. As was previously mentioned it is this simultaneous 
use of steps VI and VII that has helped develop* Metamethodology and is also 
being used quite successfuJiy in the development of the Fortune-Hutchinson 
Evaluation Methodology. 

The development of Metamethodolcgy is a significant breakthrough in 
the field of methodological research and development. It not only provides 
the procedures by which methodologiCs..l research and development are done, 
but it also provides a definition or understanding of the field. Until now 
training in the field was almost non existent, but with Metamethodology the 
training of methodologists becomes a real possibility. 

In conclusion it should be remembered that Metamethodology is not yet 
finished ♦ There are still gaps to be filled and research to be done. Some 
of the more notable gaps, although not necessarily the most important. 
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are steps II and VII. More work and further research are necessary. The 
developers, though, do believe that a workable methodology is now in existei 
and with the additional work Metamethodolony will achieve the goal of being 
able to produce the best, most efficient processes to accomplish purposes. 



Appendix 
Metamethodology 
Draft VI* 

Tom Hutchinson/ Jim Thomann 
February, 1972 

I. Put methodologist in contact with problem using one of two methods: 

A, Simple method - use interests of the methodologist 

B. Complex method - use Coffing Client - Demand Methodology 

II. State the purpose of analyzing the area and determining a purpose 
that will solve the problem. 

A. Investigate the Problem Area 

1, Read the literature in the area. 

2, Talk to people who work in the area. 

3, Examine work being done in the area. 

4, Brainstem about the Problem Area. 

5, Try out tools hhat already exist in Problem Area. 

B. Narrow down Area into manageable piece (Focus). 

C. Investigate purposes within the chosen piece of the Problem Area 

1. Brainstorm purposes that will solve the chosen problem. 

2. Read the literature applicable to the chosen problem. 

3. Ask others for purposes they think will solve the chosen 
problem. 

D. If more than one purpose has resulted from the previous step, 
then choose the most appropriate one. 

E. Check chosen purpose against following two criteria 

1. Check purpose to see that it is not trivial. 

2. Check purpose to see if it really solves the problem 
you have in mind. 

The Developers heartily thank Phillip Brooks, whose class notes were of 
great help in producing this Draft of Metamethodology. 
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3, If purpose fails to meet one of the above criteria revise 
it until it meets them both. 

F, If resources warrant show purpose to others for their critique 
based on the above two criteria. 

G. Write out purpose and commit yourself to it. [If you can say 
why you don*t like it, then you revise and recycle to E. If 
you can*t say why you don*t like it then go on to step III]. 



Ill* Test the purpose by the following criteria. 

A. Is purpose desirable? 

1. Use one of the following methods - where not obvious use 
Complex Method. 

a) Simple Method 

i) Answer question yourself with rationale 

ii) Get diverse groups to answer question 

b) Complex Method - use Coffing Client - Demand Methodology 

2, Revise the purpose if necessary. 

B. Is purpose operationalizable? 

1, Use "Operationalization of Fuzzy Concepts" 

N.B. It is not necessary to do a complete operationalization 
at this point. It is only necessary to find out if the 
purpose can be operationalized. 

2. Check A in light of Operationalization and revise if necessary- 

C. Is purpose practicable? 

1. Answer question yourself in terms of 

a) Is the development of a methodology practical given 
purpose? 

b) Is the methodology once developed a practical way to 
accomplish the purpose? 

2. Get diverse groups to answer question 

a) Methodologists answer question of (C.l.a) 

b) Methodologists and potential users answer question 
of (C.l.b.) 
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3. Revise the purpose if necessary and recycle through A 
and B otherwise go to D. 

Are existing methodolgies insufficient? 

1. Test in following way: 

a) Search area^ for existing methodologies 

b) Take found methodologies and test them against de- 
finition of methodology. If they all fail go to Step IV, 

c) Are they designed to accomplish your purpose? If not 
go to Step IV. 

d) Do any one of them accomplish you/purpose? If not 
go to Step IV. 

e) Are these practical (see if it is used) . If not go 
to Step IV. 

f) Are they desirable? If all are not, go to Step IV. 

g) Is it complete? (You may work on it if it is not). 

2. Revise the purpose and recycle through tests if necessary. 



IV. Once all answers to III are yes, then analyze implications of the 

purpose for the development of methodology (This is a vjay of identifying 
the attributes that the methodology must have.) 

A. Use following method to analyze implications (Hutchinson says 

"Problem implies its own solutions." In this case, the implica- 
tions of the purpose supplies first approximation of gross 
methodological elements.) 

1. a) Imagine and write down in what ways you could fail to 
accomplish the purpose. 

b) Imagine and write down in what ways you can accomplish 
the purpose, avoiding all the problems. 

c) Imagine the purpose being accomplished, write down 
what is happening. 

d) i] For each element determined thru b + c, determine 

all possible alternatives to accomplish the purpose. 

ii] Combine two lists into one: turn alternatives from 
a. around so they fit together with list from d.i) 

iii] Test the completeness of above list using one or 
more of the following methods. 
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1) Ask others to do steps a-c 

2) Think up alternatives which have nothing to do 
with this purpose and consider whether they do 
or not. 

3) Go back to list generated in b and c, and consider 
again whether any of those should be on list and 
add any new ones. 

4) Ask yourself if your alternatives have any 
alternatives to them. 

5) Ask what bad alternatives exist that are not 
on this list and how they could be changed to 
good alternatives. 

6) Use any other tests of your own choosing. 

2. If at this point you cannot choose one of the alternatives 
on the basis that there is some reason to believe that it 
will best accomplish the purpose then do a + b. Other- 
wise combine all lists to come out with one list - where 
there are alternatives choose one. 

a) Determine your value system. 

b) Use value system to turn list into a list of all 
positive alternatives. In other words if one of the 
alternatives is one that is contradictory or non- 
desirable use values to change it so it is not. 

B. Organize the attributes into a rational order of steps. 

1. Determine which implications are not necessary for the 
methodology (accomplishing purpose) and strike them from 
list. 

2. Determine which implications are contained in others and 
note that. Determine which implications can be combined to 
make one step, and give those a name. 

3. Ask which implications would he/she have to accomplish 
first in order to accomplish the rest. 

4. Write it out as first step. 

5. Ask which implication would now be first given the first 
one is accomplished. 

6. Write down as second step. 

7. Do this process until all major implications are accounted 
for. 
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8. 


Order 


any substeps by cycling through 3-7. 


9. 


Check 


to see if order has logical flow to it. 


10. 


Check 


to make sure ail Imp 1 ic«itf oris are stated procedurally. 


11. 


Write 


out a revised list. 


12. 


Check 


completion of ordering by asking others (at least one) 



to give an ordering of implications with explanation of why, 
if possible, without showing them his ordering. This can 
be verbal or written, depending on the resources available. 

13. Do a revised ordering based on responses from 11. 

14. Give a revised ordered list to others experienced in problem 
area and ask them to critique it. 

15. Do a final ordering and write it out. 

Add in any steps or functions that are implied by the existing 
steps at the same level of abstraction. 

Identify Anchoring Steps for Methodology 

(1, Putting Methodologist in contact with problem. 

2. Testing if methodology has worked (then recycle).) 

E. Write out final list to be used throughout rest of methodology. 

Operationalize the purpose. 

A. If the purpose is vague use "The Operationalization of Fuzzy 
Concepts", otherwise use B. 

B. Use the straight analysis technique. 



Design Procedures 

(N.B, Design or Re-design can he done at any level of breakdown 
including the highest). 

A, Identify the first (next) step to be designed (i.e. the first 
crucial step where it is not clear that the step would be easy 
to develop • Use the criteria developed in Step V to determine 
whether the step is crucial or not. 

B, Identify the step^s subpurpose. 

Analyze implications of subpurpose in terms of main purpose by 
using the procedures stated below. 



Use the following method to analyze implications of the 
subpurpose. 

1, a) Imagine and write down in what ways you could 
fail to accomplish the purpose. 

b) Imagine and write down in what ways you can 
accomplish the purpose, avoiding all the problems. 

c) Imagine the purpose being accomplished, write down 
what is happening. 



d) i) For each element determined thru b + c, 

determine all the possible alternatives to 
accomplish the purpose. 

ii) Combine two lists into one: turn alternatives 
from a. around so they fit together with list 
form d.i) 

iii) Test the completeness of above list using one 
or more of the following methods. 



1) Ask others to do steps a~c 

2) Think up alternatives which have nothing 
to do with this purpose and consider 
whether they do or not. 

3) ' Go back to list generated in b and c, 

and consider again whether any of those 
should be on list and add any new ones. 

4) Ask yourself if your alternatives have 
any alternatives to them. 

5) Ask what bad alternatives exist that are 
not on this list and how they could be 
changed to good alternatives. 



2. If at this point you cannot choose one of the alternatives 
on the basis that there is some reason to believe that it 
will best accomplish the purpose then do a + b. Otherwise 
combine all lists to come out with one list - where there 
are alternatives choose one. 



a) Determine your value system. 



b. Use value system to turn list into a list of all 

positive alternatives. In other words if one of the 
alternatives is one that is contradictory or non~ 
desirable use values to change it so it is not. 
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b. Organize the attributes into a rational order of steps. 

1. Determine which implications are not necessary for 
the methodology (accomplishing purpose) and strike 
them from list. 

2. Determine which implications are contained in others 
and note that. Determine which implications can be 
combined to make one step and give those a name. 

3. Ask which implication would he/she have to accomplish 
first in order to accomplish the rest. 

4. Write it out as first step. 

5. Ask which implication would now be first given the 
first one is accomplished. 

6. Write down as second step. 

7. Do this process until all major implications are accounted 
for. 

8. Order any substeps by cycling through 3-7. 

9. Check to see if order has logical flow to it. 

10. • Check to make sure all implications are stated 

procedurally. 

11. Check completion of ordering by asking others (at least 
one) to give an ordering of implications with explanation 
of why, if possible, without showing them his ordering. 
This can be verbal or written, depending on the resources 
available. 

12. Do a revised ordering based on responses from 11. 

13. Give revised ordered list to others experienced in 
problem area and ask th^ to critique it. 

14. Do a final ordering and write it out. 

c. Add in any steps or functions that are implied by the existing 
steps at the same level of abstraction. 

d. Identify Anchoring Steps for Methodology. 

e. Write out final list to be used throughout the methodology 

D. Determine the amount of completeness and test for it 

E. Examine the logic of the step under design in terms of subpurpose 
and main purpose. 
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F. Fill in the gaps that are found and then recycle to VI E. If 
no gaps go on to VI G. 

G. Examine the logic of entire methodology and its parts in terms 
of mair purpose in light of the step under development. 

H. Redesign step and/or methodology and recycle to VI G. If no 
gapjs then go to VI I. 

I. Recycle to VI A until one feels that further applications of 
VI will not produce sufficient improvement to warrant spending 
of resources. One may also go on to VII A as well as go back 
to VI A. 



VII. Test and then revise the purpose and/or procedures if necessary. 

A. Field test methodology; if necessary, redesign (Use Step VI), 

B. Conclusion oriented research of methodology; if necessary 
redesign (Use Step VI) . 



